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ABSTRACT 

The use of Goal Attainment Scaling to evaluate a 
home-based educational intervention in which parents acted as tutors 
of their own elementary-school-aged children is described. Goal 
Attainment Scaling is a mddel originally developed for the National 
Institutes of Mental Health as a ..ieans of evaluating the activities 
of community mental health centers. As adapted for use in a public 
school setting, Goal Attainment Seal ing proved to be a useful tool in 
not only determining program outcomes, but in also providing a 
standard for program decisions in a school environment characterized 
by flux and shifting frames of reference. How program goals were 
scaled, scored, and interpreted using the Goal Attainment Scaling 
procedure are discussed. It was concluded that the Margaret Sheehy 
School Parent Outreach Program was successful in establishing a ^ 
cost-effective hqme-based school program focusing on both academic - 
and relate'd needs of referred children. (Author/PN) 



******************************************* ^* ************** 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 
*********************************************************************** 



The Use of Goal Attainment Scaling 
In the Evaluation of a Home-Based Educational Intervention 

by 

Frank J . N-icassi o, PhD 
Child Development Program 
Umatilla Education Service District 
P.O. Box 38 
Pendleton, Oregon 97801 ^ ^ 



"PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)." 



U.S. DEPARTMENT OF EDUCATION 

NATIONAL INSTITUTE OF EDUCATION 

EDUCAriONAL RESOURCES INFORMATION 

. CENTER (ERIC) 

/ This Modintftit has bfl«n rflproducod 
(ircHivod from the pflfSon or orqfini/ntton 

Minor rh<in(}«s b*v«« boen rnilfle to improve 
fj'proflttf tion ijualilY 

• Points "f vi«w or opinions stated m ihis iioru 
mont do not nocessanly fopresent official NIE 
posit)On Of policy 



(^1 



z 



Abstract 



This paper describes the use of Goal Attainment Scaling to evaluate a home- 
based educational intervention that used parents as tutors of their own ^ 
elementary-aged school children. Goal Attainment Scaling is a TOdel orig- 
inally d.eveloped for the National Institutes of Mental Health as a means of 
evaluating the activities of community mental* tieal th centers. As adapted 
for use in a public school \etti ng , Goal Attainment Scaling proved to be a 
useful tool in not only determining program outcomes, but in also providing 
a^tandard for program 'decis ions in a school environment characterized by 
flux and shifting frames of reference. The present paper discusses how 
program goals were scaled, scored, and interpreted using the Goal Attainment 
Seal ing procedure. 
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The Use of Goal Attainment Scaling 
in the Evaluation of a Home-Based Educational Intervention 

INTRODUCTION 

Goal Attainment Scaling was used to evaluate the second phase of an 
intervention program carried ouc at the Margaret Sheehy School , Merced, Cal- 
ifornia during the 1977-1978 academi c year. Sheehy, located in the agri- 
culturally rich San Joaquin Valley, served 640 children in a kindergarten 
through fifth grade school setti ng characterized by high student turnover, 
low socioeconomic indices, a high proportion of culturally non-dominant fam- 
ilies, and low achievement test scores (State of California, 1977). The Parent 
Outreach Program (POP) was one component of an overall effort Committed to 
raising the educational performance of the large proportion of children who* 
scored within the first quartile of the Stanford Achievement Test. Based on 
indicators in the. educational literature (Nicassio, 1977a), POP assumed that a 
child's school performance could be improved by grooming the child's parent to 
become a primary agent of educational intervention. It was proposed that this 
step could be accomplished by having the parent, with the assistance, direction, 
and support of a representative of the school, tutor the child at home. 

The program evo'lyed through successive stages and began in 1975 by adapt- 
ing materials from Project Home-Base, a federally sponsored preschool program 
(Yakima Public School, 1975, 1976) which was modeled after the Gordon (1969) 
Infant Stimulation Project. In 1977 an attempt was made to expand the home based 
curriculum, provide supportive education for project staff, clearly delineate 
goals, and evaluate outcomes. The writer was enlisted as program consultant 
to aid in these tasks . 

Four strategies for achieving a set of broad goals formulated by the faculty 
and the School Advisory Committee can be briefly summarized: 

1) Use paraprofessi onal Parent Educators to facilitate a coordinated 
and collaborative home and school study program by means of 
weekly home visits . 

2) Provide Home-Tasks (activities focusing on self-help, socialization, 
perceptual -motor , cognitive; language, reading, and math skills) 

for use by parents with their children. 

3) Instruct parents in the use of ten Desirable Teaching Behaviors 
(DTBs) to improve instructional ly relevant interactions between 
parents and children as they work together on Home-Tasks. 



4) Have Parent Educators use and model for the parent, a problem 
solving strategy to improve communicative interactions between 
parents and chi 1 dr^n as they contend with the often chronic, 
pervasive, and globally ill-defined day-to-day problems of 
interpersonal 1 i vi ng. 

These strategies became the basis for the objectives specified on Goal 
Attainment Guides (Tables 2, 4, 6, 7). 

Intervention was carried out by three Parent Educators (ni cknaiTied PEs or 
POPPEs) who designed a home-study regimen for each of 39 families that integrated 
concerns identified bv the child's parent, the classroom teacher, and the child. 
Significant aspects of each home-visit were recorded on a structured log sheet, 
adapted from Project Home-Base (Yakima Public Schools, 1975^ 1976), called the 
Parent Educator's Weekly Report Guide (PEWRG). These data provided much of the 
information used in measuring program outcomes. A complete description of 
demographic variables, family selection niethods , program activities, etc. is 
contained in POP's recent evaluation report (Nicassio, 1978). 

THE EVALUATION SCHEME 

Early in the program, the coordinator attempted to design an evaluation 
scheme that could be termed "traditional," "classical," or "research oriented." 
Given program constraints that made it impossible to select experimental and 
control children randomally from a pool of referrals, or to even establish 
"waiting list controls," the "traditional" hypothesis-testing approach was 
abandoned. A subsequent attempt to integrate quasi-experimental design (Campbell 
and Stanley, 1963) with single-subject design (Hersen and Barlow, 1976) was also 
abandoned. Although such an integration holds considerable promise for answer- 
ing the educational questions posed by school personnel, more time and effort 
were required than was available in the working regimen of Parent Outreach. 

It has been rfecognized that the problems of evaluation frequently differ in 
intent from heuristic research (Weiss, 1972, p. 6), and that evaluational settings 
do not necessarily provide the conditions for formal experimentation (Campbell 
and Stanley, 1963'; Pppham, 1975 , p. 4). Although a "classical" approach was not 
possible within the constraints of the POP environment, this did not obviate the 
need for evaluation of program outcomes. A ready audience for evaluational results 
was provided by the faculty and the School Advisory Committee (comprised of 
faculty and parents) whose continued allocatfion of financial support was contin- 
gent on program outcomes that could be monitored by them daily, on an informal basis, 
in the classroom and in the home. 



If one assumes tha^ a valuable r6£ulth.of program evaluation is informa- 
tional feedback upon whi(^h changes may be introduced^ to enhance program 
effectiveness, then many -suffi cient evaluatign models dg exist (Popham 1975 ; 
House, 1978). Stuff lebeam's CIPP (Stuff lebeam et al . , 1971), UCLA's CSE (Alkin, 
1969), and ^various goal, attainment models' are examples (Popham, 1975). Each of 
these orientations enable program staff to make decisions that are sensitive to 
accountability and productivity. The particular system called Goal Attainment 
Scaling (GAS), developed by Kiresuk and Sherijian (1968) for/ the National 
Institutes Mental Heal th ^s a means of evaluating the activities of Community 
ilental Heafth Centers was selected because of its ability to validly maintain 
empirical structure for activities occurring within the environment of a fluid 
and enfolding organization. It should be noted that GAS has oeen central to many 
earlier efforts in both work with children (Byassee and Tamberino, 1975-76 ; 
Garwick, 1976; Hegion, Fish, and Grace, 1974; and Stoudenmire sndComola, 1972) and 
in evalua/tion "of program goals (Kirkhart, 1977; Stelmachers, Lund, and Meade, 
1972; an/5 Webb, 1975-76) . 

Ba/ed on the broad goals set'by the faculty and School Advisory Committee 
and. the tour strategies cited above, measurable objettives were writtent^/or four 

im a\eas labelled Parent/Home, Student, Teacher/School, and Administrative/ 
nance. These objectives were then "scaled" on Goal Attainment Guides 
(Tables 2, 4, 6, 7) which were modified for use within a school setting. This- 
pi"0cess, which required consultation with school and project staff, was carried 
out over a two day period in September 10:^7 by the program consultant, prior to 
POP's school year activities. 

"Scaling" an objective means that both positive and negative outcomes of the 
program's efforts are anticipated in advance of their actual measurement at pro- 
gram's end. For example, looking at tKe Goal Attaiwnt Guide titled "Adminis- 
trative/Maintenance" (Table 2) we see tlie first columi\^is labelled "Scale 
Attainment Levels" and shows five possible outcomes: +2 is "Best Anticipated 
Educational 0 utcome' Thro ugh t Likely;" +1 is "More Than Expected Level of Educa- 
tional Success," etc. The second column _of this Goal Attainment Guide is labelled 
"Scale 1: Home-Task File." The 0, or "Expected Level" portion of this column 
states an objective concerning the development of Home-Tasks. . Given program 
resources and constraints, it was anticipated that POP could develop 100 Home- 
Tasks by the time of follow-up in June 1978. Any number of tasks fewer than 100 
was considered a "Most Unfavorable" outcome and was so stated on the Goal Guide; 
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any number greater than 100 was eonsideredt better than expected and was so 
scaled. It was also judged that .the " creatipnof a Honie-Task fi l^^was signifi- 
cant enough at this stage of program development that it should be weighted 
heavily with respect to other objectives. Scale 1 was therefore , given a 
weight of W^ = 40 (on a 10, 20, 30, 40 50 scale) and so noted at the top of the 
column. This weight was used in subsequent computations. At follow-up, the 
number of Home-Tasks completed was counted and the result of 145 was indicated ' 
by placing a double star symbol in the appropriate box. This objective thus" 
earned a raw score of +1 ("More Than Expected Level of Educational Success") 
or a weighted raw score of ( + 1 )X(40) = +40. Other scaled objectives covering the 
four program areas were treated in a similar manner by scoring them for level of 
attainment. Weighted raw scores were transformed to normally distributed T-scores 
with a mean of 50 and a standard deviation of IO5 accord-u)g to the computational 
method devised by Kiresuk and Sherman ( 1968). Percentile ranks were then derived 
using sthe conversion table prepared for this purpose b/'Garwick (1971). 

Goal Attainment T-Scores were computed using the data sources specified on 
the Goal Attainment Guides.. These included questionnaires, frequency counts, check- 



lists, school test results, and especially the PEWRG menti oned* above. Reliability 
of measures recorded on the PEWRG were verified by the program consultant's 
intermittent home-visits wi th Parent Educators. Since data were collected over 
the course of nine months, it required th*^ last week of the school year for the 
POPPEs and program consultant to validate <ind summarize data and to compute 
the results described below. 



The results of POP efforts may be looked.,at in terms of Goal Attainment 
T-scores for each of the four program areas and for the program as a whole. 
These outcome scores, and their corresponding percentile ranks, are shown in 
Table 1. 

The percentile rank is interpreted as follows: Given* all possible outcomes 
of the objectives scaled on the Goal Attainment Guides, the obtained outcome 
(T-score) represents a score such that X% of the others fall belo-w it. For 
example, the outcome earned by POP for the attainment of all 20 program 
objectives subsumed under the four program areas was a percentile rank of 82. 
Thus, of all possible outcomes stated on the Goal Guides, POP achieved better than 
82% of the other possibilities. This is a high yield for program efforts. 
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Table 1 

GAS outcomes for each- of four program 
components and for the program as a whole 



Goal Attainment Percentile 
Program T-Score Rank Attained 

Aciminlstrati ve/riaintenance 47.4 40 ^ 

(10 seal es ) 

School/Teacher 70.-0 . 98 

(3 scales) 

Student 50.8 54 

(2 scales) \ 

Home/Parent 62.2 \ 88 ^ 

(5 seal es ) 

Overall 59.4 82 

, (based on all 20 s cales ) 

If we interpret these results in the manner described, and note that 
a percentile rank score of 50 represents the exact attainment of a specified 
objective, we can pronounce POP as having been a success for the school year 
since it not only met its criteria of success, but surpassed its expectations 
for all but a single program area. 

These results also have an alternate interpretation. That is, scores falling 
either far below or far above 50 represent objectives that may be either too 
difficul tly or too easily attai^ned given the characteristics of the particular- 
program. Consequently, "School /Teacher" showing an outcome of 98 may represent 
a set of program objectives that have initially been set too low, while 
"Administrative/Maintenance" with an outcome of 40 may i<»epresent a set of objectives 
that have initially been set somewhat high. A strength of GAS is that. such feed- 
back provides program staff-with information relevant to adjusting criterion levels 
on subsequent cycles of program delivery and evaluation. .In order to implement 
changes in these levels, however, one must consider the nature of specific 
objectives contained within each program area. This is the task to which we now 
turn. 
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A dmini strati ve/Ma intenance Component 

This program area concerned itself with the establishment and operation of 

programmatic necessities: Ten objectives were scaled (Table 2). 

Scale 1: Home-Task File: It was anticipated that 100 Home-Tasks 
could be constructed and categorized during the academic 
year. One hundred forty-five were produced placing POP 
above the expected level. The creation of Home-Tasks 
was weighted in importance at a high 40, since the object 
of intervention was to facilitate parent/child interaction 
around these materials. It is unlikely that so much 
importance would be placed on the creation of curricular 
materials in subsequent year^. Consequently^ fewer 
numbers of tasks and a lower weighting would be appro- 
priate for future cycles of program operation and evalua- 
ti on. 

Scale 2: Translation: POP achieved below expectation and trans- 
lated only 48% (n=70) of the Home-Tasks into Spanish. 
This presented no particular difficulty for delivery of 
services to bilingual or non-English speaking families. 
Thus, quantitative results defer to qualitative inter- 
pretation. A future objective should be to translate the 
remaining tasks for bilingual use. 

Scales 3 and 4: Coordination and Information Dissemination: 

These objectives represent an effort to keep channels 
of communication open and to insure consensual under- 
standing on the part of all POP staff. There were a 
total of 35 coordination meeting's between PEs and the 
coordinator, and a total of 13 information memos which 
summarized procedures^ concerns, interim results, goals, 
etc. The yield was high as evidenced by statements of PEs 
citing coordination meetings as being especially beneficial 
to decision making in the field. Future POP efforts 
should continue to incorporate high levels of staff 
i nte r change, 
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ORGAN. Sheehy 



PROGRAIiJ OR Parent Outreach Program . 
COMPONENT Administrative/Maintenance Componentl 



I 



START 
DATE 



SePt 1977 



FOLLOW-UP page 1 of 2 
DATE June 1978 



G 0 Al 



A T T A I N M E fJ T 



GUIDE 



SCALE ATTAINMENT 
LEVELS 



SCALE! : Home-Task 
(W^- ho) File ^ 



SCALE2 :Translation 
(W2= 30) 



SCALE 3 : Coordination 
(W3= ^0) 



SCALE 4: Information 
(W/|= 10 )Dissemination 



SCALES :Home Visits 
(W5= 50) 



f 2 

Beet Anticipated 

Educational 

Outcome 

Thought' Like ly 



151. 



200 tasks 



More than 85^ 
of tasks 



Moi'e Than 
Expected Level 
of Educational 
Success 



101 - 150 tasks 
lie* 



8U - 85>o of tasks 



Expected Level 
of Educational 
Success 



Establish a Home-Task 
file of 100 tasks by 
June 1978 as evidenc- 
ed by numer i ca 1 count 



Translate 80?; of ^ 
tasks i nto Span i sh 
by June 1 978 as 
ev i danced by nume r i 
cal count. 



t 

Hold more than 
20 meet i ngs 

tic 



Hold 20 meet! ngs 



More than 1 0 memos 



Hold at least 15 POP 
staff coord i nat ion 
(meetings as evidenced 
bv mi nutes or notes 
by June 1978. 



POP consul tant will 
produce at least 10 
i nf or ma t ion summa ry 
memos by June 1978. 



Complete more than 
80% of possible 
home visits 



Complete at le-::st 
80% of al 1 possi bk 
home visits per 
family as recorded 
on PEWRG II and ill 
by June I978. 



- I 

Less Than 
Expected Level 
of Educational 
Success 



- 2 

Most Unfavorable 
Educational 
Outcome Thought 
Like ly 



Hold fewer than 
1 5 meet i ngs 



Complete 60 - 79^ 
j of poss i b 1 e home 
I visits 



Any number of 
tasks fewer than 
100 



Less than 80% 
of tasks 



PROPOSED PLAN 
OF ACTION 



o 10 
> ERLC 

fin 7-17-72 



Parent Educators 
(PE ' s ) and consu 1 tant 
wi 1 1 develop tasks as 
needed . 



Table 2: Goal 



Att^inm^nt 



Bi 1 ingual PE's 
wi 1 1 trans 1 ate 
tasks as requi red , 



Gu i de for the AdmTi 

f 



Consul tant wi 1 1 org- 
an) ze meetings based 
on needs ident i f i ed by 
us ! ng V i sua 1 display 
Dyad Weekly Outcome 



Fewer than 10 memos 



Complete less than 
60% of possible 
home visits 



Reco rd notes and 
memos and maintain 
f i 1 e. 



1 i strat i ve/Ma i ntenance Component 



Sustain home- v i s i X 
program and record 
results on Parent 
Educator ' s Weekl y 
Report Gu i de 
(PE\^RG) . 

li ^ 



A PROGRAM OR Parent Outreach Program START FOLLO'w-UP page 2 of 2 

ORGAN. Sheehy - COMPONENT Administrative/Maintenance Component DATE Sept 1977 DATE June 19B' 



GOAL 



ATTAIN ME rjT 



GUIDE 



SCALE ATTAINMENT 
LEVELS 



SCALE 6 : PE 

j;w =10 ) Presentation 



SCALE 7 : PE 
(1^/7= 20) DTB's 



SCALE 8 : Task 
(l^= 20 ) Selection 



SCALE 9 .• 

2Q ^ Counsel! ng 



•f 2 

Best Anticipated 
Educational 
Outcome 
Thought Likely 



Two or more methods 
of task presentation 
on more than 75% of 
completed home visits 



Ei ght or more DTB ' s 
on more than 75% of 
completed home visits 



Parent responds 
more than 75% of 
time with #3 



On b^% or more 
home visits 



of 



SCALE la Length of 
(1^0= 20 ) Home Visit 



More than 80% 
of f ami 1 i es 



^ I 

More Tnan 
Eoq^ected Level 
of Educational 
Success 



Expected Level 
of Educational 
'Success 



PE uses two or more 
methods of task pre- 
sentation on 75% of 
completed home visits 
as recorded on PEWRC 
V, by June 1978. 



E uses 3 or more DTG 
on 75% of completed 
nome visits as record 
ed on PEWRG VI , by 
June 197G. 



Parent wi 1 1 respond 
wi th a ^3^ 75/0 of ^ 
the time as recordedj 
on PEWRG X, by June I 
1978. 



Parent received other 
than task information 
on 50% of home visi ts 
as recorded on PEWRG 
XII ,XIII ,XVI by June 
1970. 



! 



80% of families 
average at least 
min. of PE contact 
time per home visi, 
by June 1978. 



I 



- I 

Less Than 
Expected Level 
of Educational 
Success 



Two or more methods 
of task presentation 
on 51% - 74% of 
completed home visits 

tA 



Ei ght or more UTB ' s 
3n 51% - 74% of 
compl eted home vi s i ts 



Most Unfavorable 
Educational 
Outcome ^-Tlic ugh t 
Like ly 



T//0 or more methods 
of task presentation 
on fewer than 51% of 
completed home visits 



i ght or more DTB ' s 
on fewer than 51% of 
completed home visits 



Parent responds I 
51% - 74% of time 
v/ith ^3 



Parent responds 
less than 51% of 
time with if 3 



On less than 50% of 
home visits 



Fewer than 80% 
of fami l ies 



PROPOSED PLAN 
OF ACTION 

ERslC 

"TjrTA 17-72 



Sustain home visi t 
program. Monitor Dyad 
l\ieekly Outcome on 
PEWRG. 4n-service of 
deficits by consultant 

Table 2: (continued) 



Same as Scale 6 



Same as Scale 6 



Same as Scale 6 



Same as Seal e 6 



13 



00 



9 



Scale 5: -Home Visits: Since PEs were expected to model con- 



structive interaction patterns for parents, a sustained 
and high frequency of home visits was seen as critical 
to the eventual improvement of a child's school achievement. 
POP fell considerably short of the expected 805^ comple- 
tion of possibleXome-visits by logging only 339 completed 
home visits during 580 family-weeks of activity. This 
represents a 585^ completion rate (339 -f 580 = 58/0- 

In general, this low showing resulted from several 
unforseen circumstances. Th^firstwas the high frequency 
of task avoidaniseJi^ famil ies during their early weeks in 
the program. For example, a new family would complete 
the first visit or two and then break several subsequent 
commitments, only to pick up on vis its again at a later 
date. One inference* is that the time-span represented 
by missed visits is actually a latency period in which 
Home-Tasks, DTB's, and the rest of POPs basically middle 
class devices were accommodated to a family's personal 
and cultural life-style. Since the simple translation 
of carricular material such as Home-Tasks into either 
Spanish or the vernacular does not guarantee cross- 
cultural sensitivity or. equi val ence , PEs were encouraged 
to empathetical ly persist in the face of this ci rcumstance 
so as to gain greater insight into inter-cultural assimi- 
1 ation. 

Another circumstance related to the above were 
families requiring numerous "get acquainted" phone calls 
or "front porch" conversations that functioned to allay 
anxieties associ ated w'i th formal looking and sounding 
school materials. This situation was best summed up by 
the PE who commented that, "It took me six months to 
get invited into the house." 

Still another circumstance included several families 
who were contacted only by phone with regards to some 
particular concern. For instance, following a dU€fcp,^^__^ 
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the' affected child's classroom teacher sought the assis- 
, tance of POP to provide supportive counseling. Over a 

21 week period^ jo^ home visits were completed, yet the 
PE contacted the parent four times by phone, and the 
child'sixty-three times at school. These sixty-seven 
^contacts enhanced the parent/child/school relationship and 
quite likely supported the child's school efforts although 
no data exist to support this contention. 

The low showing for scale 5 is also a result of the^ 
manner in whiCh the outcome was computed i.e., only 
completed home- contacts were considered. If,jyowever, 
the number of phone contacts (351 contacts totaling 32.3 
hours of parent/PE conversation) and the number of other 
contacts- at school, at the PEs home, or on the street 
(116 contacts totaling 39.9 hours of parent/PE discus- 
sion) were combined with completed home-visits (339 home- 
visits totaling 181.4 hours of parent/PE interaction) POP 
can claim 806 direct parent contacts or 1.6 contacts 
(806 508 family weeks) per family per week in the pro- 
gram. Communication between home and school was obviously 
kept open. 

It is recommended that future statements of objec- 
tives take into consideration the family contacts that 
occur outside the formal home-visit and use such informa- 
tion in the computation of evaluational outcomes. 

Scale 6: Parent Educator's Presentation: This objective 

attempted to address the variety of task presentations 
that PEs modeled for parents (expl anation , >ole-pl ayinvj, 
demonstration, listening to parent make presentations). 
In 64% of the home-visits, PEs utilized Wo or more methods 
of presentation which fell somewhat short of the 75% level 
anticipated. Since this represents no Critical deficit, 
it is recommended only that continued attention be paid 
to this area during subsequent in-service training 
sessions. 

o 
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Scale 7: Parent Educator's DTBs: Thi s 'objecti ve gauged the use 
of Desirable Teaching Behaviors during home visits. 
Parent Educators utilized 8 or more DTBs on 66% of their 
completed visits* This result may be interpreted in a 
manner similar to that of Scale 6; a like recommendation 
is al so warranted. 

Scale 8: Task Selection: This scale metered parents' opinion of 
the difficulty of Home-Tasks for their children. A 
full 85% of the tasks were considered to be "just right 
for the child" as recorded on the Parent Educator's Weekly 
Report Guideo Future programs should continue to encourage 
the collaborative planning with the teacher and parent 
concerning selection of Home-Task materials; this should 
insure that task selection remains a strong point in the 
program. 

^ Scale 9: Counseling: In 93% of the completed home-visits, parents 
received assistance with other than educational concerns. 
Because of behavioral, emotional, or other family 
difficulties, PEs provided information on welfare and 
community services, child growth and development, health 
care and nutrition (one PE even became involved in an 
obese child's prescribed weight loss program), and enrich- 
ment materials'?* Thesd areas represent pressing needs 
within many families and must often be dealt with before 
parent or child can establish an environment conducive 
to home study. 

Parent Educator efforts in this non-academic area 
probably have long-run, although educationally unmeasur- 
able, effects on families. For instance, a parent whose 
interaction with her child precipitated "school phobia" 
was given relevant reading material (Patterson and Gullion, 
1971) and put in contact with a counselling information 
center at Merced College. After encouragement, twq other 
parents began attending an evening class in "Systematic 
Training for Effective Parenting" held at Sheehy by the 
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school psychologist. Other parents were 
assisted in obtaining eye glasses for their children, 
or clothing, or other neQe^^sTties. Two parents were .f 
assisted with enrollment /nd attendance at Merced College, 
while still t^^^o others were aided in obtaining necessary 
ernploVment. Three POP parents now volunteer at school: 
one works in the, library, one in a classroom and the 
third serves commendable on the School Advisory Committee. 
Each of these results seems directly attributable to 
the patient, considerate, friendly, and interested 
counselling carried out by Parent' Educators (although 
the usual cautions regarding causal inferences should 
be observed). These improvements cannot be expected 
to show up in increased SAT scores in a single year, 
but can be thought of as " faci 1 i tatCFrs" that, if sus- 
tained, should enhance the developmental outcomes of 
the children served. 
Scale 10: Length of Home Visit: Only 59% of the families averaged 
PE contacts that we 30 or more minutes in length. 
The distribution for the thirty-nine POP families is 
shown in Table 3. This outcome is difficult to inter- 
pret since there is no known function between length 
of home visit (or quality of home visit) and program 
output. The variable results of Table 3 coincide with 
findings of the National Home Start Evaluation (Love 
et al., 1976, p. 79). The data are included^^re as 
empirical fact that may be useful in establishing / 
"baseline" for future activities. 
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Table 3 

Number families receiving home visit 
contacts of varying durations 

Average Length of No. of 

Contact in Minutes Fa mil ies 

60-120 5 
50-59 ^ 4 

40-49 4 
30-39 10 
20-29 3 
10-19 n 
0-9 2 

School /Teicher Component 

This program component concerned itself with the elicitation of teacher 
participation and of teacher response to the program effort. Three objectives 
were scaled (Table 4); 

Scale 1: Target Specificity: In order to establish the coordinated 
and collaborative home and school intervention required by 
program goals, PEs attempted to define, with the referred 
child's classroom teacher, a statement of the academic 
problem in specific enough terms to guide selection of 
Home-Tasks. This is not as simple a job as it appears, 
for all of the affected parties (child, parent, 
teacher, PE) must arrive at a statement that recognizes 
the divergent interests of each. For example, 
although the cl ass room' te aether may recognize "visual 
and auditory discrimination" as^deficit areas, the 
parent may recognize behavioral oiHbursts as salienj 
whif^ the child may not even be abl e^O verbali . 
feelings of discontent and malaise. The kE7 on the other 
hand, may become biased by the existing file-of Home- 
Tasks and thus interpret chi 1 d/paVent/ teacher concerns 
in. terms of available curricular materials. (The evidence 
actually suggests that PEs, during this year of intense 
Home-Task construction, were quite flexible in task 
selection). A first step towards the collaboration of 
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PROGRAM OR Parent Outreach Program . START FOLLOW-UP page 1 o'f 1 

ORGAN. SheHiv ' COMPONENT School /Teache r Component DATE Sept. 1977 DATE June 1978 



GOAL ATTAINMENT G U 'I D E 



SCALE ATTAINMENT 
LEVELS 


SCALE! : Target 

(Wp 20) Specificity 


SCALE2 : Unsolicited 
[Wr^ 50) Responses 


SCALE3 : Reaction 
(W3= 50)Sheet 


SCALE4 : 
(W/,= ) 


SCALE 5: 
(W5= ) 


-h 2 

Best Anticipated 
Educational 
Outcome 
* Thought' Likely 


More than 902; nf 
part i c ! pat i ng 
teache rs 


More than 50% of 
part i ci pat i ng 
teachers 


More than 80% will 
respond favorably 


1 




^ I 

More 'Than 
Expected Level 
of Educational 
Success 






i 


i 




0 

LJ JL>L/ t> O L/' t> ^ I_/C> L/t> U 

of Educational 
Success 


30% of part i c i pat i ng 
teache rs will have 
i dent i f i ed & spec i f i - 
eu d LargeL prooiern 
us i ng Behav i ora 1 
Anal ys i s and record 
prob lem on back of 
Teacher Check-list by 


50% of part i c i pat i ng 
teache rs will make 
an unsol i c i ted and 
po s i t i ve comnne n t 
abou t ch i 1 d ' s pro- 
gress i n POP as 
recorded i n Comments 
section of PEWRG by 


80^^ of part i ci pati ng 
teachers wi 1 1 respond 
favorably to POP on ar 
end of program 
React i on Sheet . 


- 

1 




- I 

Less Than 
Expected Level 
of Educational 
Success 


June 1978. 

50^ - 89% of teachers 

* K 


June 1978. 


i 
i 

! 


i 
t 

i 


- 2 

Most Unfavorable 
Educational 
Outcome Tiiought 
Like ly 


Fpwp r f ha n ^09^ nf 

part i c i pa t i ng teache r 


rewer tnan ^\j/q or 
^ part i ci pat i ng 
teachers . 


Fewer than 80% will 
respond f avorab 1 y . 


i 
1 




■ PROPOSED PLAN 
OF ACTION 

ERIC 

"fS™7-17-72 


i n-s erv i ce of PE ' s in 
Behav i ora 1 Ana 1 ys i s 
by consu 1 tant . 

in-service of teacher 
by PE. 

Table h\ Goal Atta 


Hon i tor PEWRG 

'nment Guide for the S< 

/. ._. 


Develop and admin- 
ister Reaction Sheet 
by June 1978. 

::hool /Teacher Component 


1 

1 * 


20 - 



15 



all parties is ,to assist each in stating concerns 
specifically. With regards to classroom teachers, an 
attempt was made to glean information that could be 
used to synthesize a treatment approach by means of 
a problem solving strategy callefl the RORPE Field 
Analysis (Nicassio, 1977c). Although ule of the POPPE 
Field Ar^lysis proved less than adequate to the^ ta§k, 
PEs were**able to get 82% of the participating teachers 
to define a child's academic concerns with enough 
specificity to guide treatment. This is somewhat 
short of the 90% expectation but would seem to indicate 
a certain modicum of success, nonetheless. A strategy 
for updating academic target areas was a weekly report 
memo sent to classroom teacjjers by the PE. This 
'device got off to a slow start but increased in 
popularity after a "reply" portion was added to the 
memo. This system of teacher/PE communication should 
continue to grow in usefulness in future years. 

Scale 2: Unsolicited Responses: Indicative of a program's 
progress are the unsolicited comments made about 
discernable changes observed in affected (Children. 
A full 93% of participating teachers made unSjOlicited 
and positive comments about POP children. Some of 
these testimonials follow: 

has improved greatly since 



the family (grandmother and uncles) is 
working with him." 

" is showing more effort. It's 



much better now." 

" is more independent." 

"Tm pleased." 

" will try more now." 

"I've noticed an improvement." 

Although testimonials are subjective impressions, they are 
among the firs*', indicators to signal changes in chil- 
^ dren's behavior. Opinion plays an important role in 
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the labelling of children as potential beneficiaries 
of special programs (e.g., E.M.R., L.H. , M.G.M., 
P.O^P.). There is no reason to take opinion less 
seriously when it comes to restorsing children to the 
ranks of the "educable," 'V?ll adjusted," or "normal," 
particularly in light of what has been termed the 
Pygmalion Effect (Rosenthal and Jocobson, 1968; Brophy 
and Good, 1974, pp. 30-41.) One need only remain 
cautious in attributing too many of the noticed be- 
havior changes to the influence of POP; that is con- 
servative, but a necessary caveat , nonetheless. 
Scale 3: Reaction Sheet: A questionnaire was given to teachers 
during a faculty meeting at the end of April 1978. 
Responses were summarized along with qualitative 
replys addressing sources of satisfaction and frus- 
tration with POP. Recommendations for improvement 
were also made. The complete results were distributed 
in a memo to the faculty and School Advisory Committee 
dated April 27, 1978. The tabulated portion of the 
reaction sheet is reproduced below as Table 5. The 
results differ slightly from those presented in the 
memo since a subsequent questionnaire was turned in by 
a teacher who had been absent from the faculty meeting. 
In general, results indicate that teachers were sat- 
isfied with Parent Outreach. This attitude is also 
reflected in a faculty vote which allocated funds for 
the 1978-1979 school year; eighty-seven percent of those 
polled (13 of 15) voted to continue and expand POP 
while 13% (2 of 15) voted to modify it. 

Some of the weakness in operating procedures that 
were noted by teachers included: less than optimal 
coordination with classroom programs; less than clear 
feedback about the character of home visits and 
parent/ child attitudes; concerns over the ethnic/ 
racial representativeness of families served, of PEs , 
_ and of the coordinator; and concerns that POP was not 
able to cope with the many crisis referrals, and with 
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SHEEHY /SCHOOL POP 



TEACHER'S INDEX 



(NO NAME PLEASE) 
Date N = 20 



(Place an X Jn the column i>Tsilcating your 
reaction to the question) 

1. The POP Program was able to achieve most 
of its goals as stated at i ts • i qforma t i on 
meeting in November, 1977» 

2. The POP^program fit well into the 

overal 1 school program. \^ 

3'.' I was able to keep informed as to the 
operation of the program by rrl^ans of 
the weekly POP Report to Classroom Teachers. 

h. POP provided appropriate school 
experiences for children at home. 

5. POP was successful in coordinating its 
activities with the classroom. 

6. POP was successful in its work with 
children in their home. 

7. Specify the degree to which you feel 
you were involved In POP, 



8. 

9. 
10. 



NO 

RESPONSE 


POO 
1 


R 

2 


3 


EXCEL 
h 


LENT 
5 ] 


NO 
CONTACT 


2\% 






32^0 


26% 




21% 








30% 


35% 


15% 


5% 




5% 


5% 


25% 


35% 


5% 


25% 


5% 






30% 


25% 


10% 


30% 




1 0% 




25% 


20% 


5% 


30% 


5% 


10% 




35% 


25% 




25% 


\5% 




30% 


25% 


10% 


10% 


10% 



Did POP provide information about children useful in your classroom 
planning? yes no 



Were chi 1 d ren frorfi your classroom involved in POP? 



yes 



no 



List any sources of satisfaction the program prov ided. 



11. List any sources of frustration. 



12. How might the program be enhanced? 



Table ^: Faculty response to Parent Outreach efforts, April 1978. 
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other families requiring assistance. Many positive 
aspects were also noted on the above mentioned memo. 
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Student Component 

This area concerned itself with the frequency of Home-Task Completion, and 
with student affective response. Two objectives were scaled. Each was weighted 
quite heavily (Table 6). 

Scale 1: Home-Task Completion: This scale gauges the extent of 
student involvement in Home-Tasks. It was anticipated 
that 80% of the participants would cover at least one 
task for each two weeks in the program. The result 
of 58% falls below this expectation and earned POP a 
negative one score for a goal weighted at 40. POPs 
deficit functioning is here related to the problems dis- 
cussed above with respect to "Administrative/ 
Maintenance: Scale 5: Home Visits," and will not be 
repeated because the situation was already best 
characterized by the PE who commented on the six months'-; 
of work required before being invited into the home 
during the 1976-1977 school year. The PE persisted and 
her notes go on to record that, "I slowly gained her 
confidence. Last year our POP visits were more regular 
after I g^iined entry into the hore." 

Scale 2: Student Response: This scale attempted to detect 

changes in student affect. It was weighted at a maxi- 
mum of 50 because positive affect is believed to be 
central to the optimization of a child's academic and 
developmental well being. It is gratifying to report 
that 64% (n=25) of participating children made unsolicited 
comments to PEs that could be interpreted as enthusiastic 
acceptance of the honne visit regimen. Comments usually 
focused on the Home-Task, e.g., the child would stop the 
PE at school and tell her that the work that had been 
brought the night before was already completed, or the 
child would ask the PE when she was going to bring more 
work to the home, or the child woUld say it had been 
fun doing the task with the parent; Parents also made 
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) PROGRAM OR-Parent Outreach Program START 
ORGAN. Sheehy COMPONENT Student Component DATE S( 

GOAL ATTAIN ME r^T GUIDE 


FOLLOW-UP page \ of i 
2Pt. 1977 DATE June 1978 


SCALE ATTAINMENT 
LEVELS 


SCALE 1 : Home-Task 
(Wp ko) Completion 


'SCALE 2 : Unsolicited 
(W2= 5o) Responses 


SCALE 3 : 


SCALE 4 : 
i\- ) . 


SCALE 5 : 
(W5= ) 


2 

Best Anticipated 
Educational 
Outcome 
Thought' Likely 


More than 80% of 

pa rt i c i pa t i ng ch i 1 d- 

ren 


More than 50% of 
part i c i pat i ng 
ch i 1 dren 


i 


1 




^ I 

More Than 
Expected Level 
of Educational 
Success 






i 

i 
1 

j 




0 

Expected Level 
of Educational 
Success 


80% of part i c i pa t i ng 
ch i 1 dren will have 
worked thru 20 Home- 
Tasksr^Tor a hO week 
period (prorata adj- 
ustments) as recorded 
on PEWRG i V, Vi i , IX, 
bv June 1978. 


50% of part i c i pat i ng 
ch i 1 d ren will make an 
unsol i ci ted and pos- 
i tive comment about 
POP home vi s i ts to PE 
as recorded i n Commen 
section of PEWRG by 
June 1978. ^ 


i 
1 

1 

1 




- I 

Less Than 
Expected Level 
of Educational ' 
Success 


50% - 13% of parti- 
ci pat i ng ch i 1 d ren 




t 
1 

1 

1 




^ 2 

Most UnfavorobJ^ 
Educa tiona^^ 
Outcome T^qught 
Like ly ^^n^^ 


Fewer than 50% of 
part i ci pat i ng ch i 1 d- 
ren 


Fewer than 50% of 
.part i c i pat i ng 
child ren 




1 


1 


PROPOSED PLAN 
OF ACTION 

ERLC 

fin 7-17-72 > 


Monitor PEWRG. 
Pers i St at eliciting 
parent/ch i'^l d coopera- 
1 1 on. 

Table 5: Goal Attaii 


Monitor PEWRG 
•ment Guide for the 5t 


0 

L dent Component . ^ 


1 

I 


26 --^ 

V 
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unsolicited comments indicating their satisfaction 
at being assisted in the non-threatening environment 
of their home, to help their own children. Spanish 
speaking parents were especially pleased at learning 
how to help their child in their native language. 

Changes in ^rtti tudes of participating family 
members toward school activities is considered a 
central purpose of POP. That such a large number of 
POP children freely commented on their high motiva- 
tional level is interpreted as a sign of success. 

Home-Parent Component 

This program area focused on refining parent/child educational interactions. 
By structuring parent participation at home, improvement in interactive quality 
was expected to occur. Five objectives were scaled for evaluation (Table 7). 
Scale 1: Target Specicity: This scale is related to a similar 
objective for teachers. Here, PEs attempted to help 
parents focus on a single, specific academic area of 
interest; one that could be carefully obs^erved for 
baseline and intervention effects. Parent Educators 
have indicated that this endeavor was no average 
challenge. One RE observed that, "Parents can't come 

o 

up with goals because they don't know how to observe 
their kids." This comment echoes a precondition to 
any form of quality intervention~~good observation. 
Informed and critical observational ability is not 
wanting in parents alone, however; it is a deficit that 
pervades educational endeavors at all professional 
levels. It is noteworthy that of the 21 families con- 
cerned primarily with their child's academic abilities 
(behavioral and counselling concerns excluded), 81% 
(n=17) were able to verbalize and target academic goals 
on which to concentrate home effortsfonce assisted by 
the PE. Aiding parents in structuring objectives for 
their children and themselves should be vigorously 
pursued in further replications of Parent Outreach. 
This is one important step towards self-determination. 
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"X PROGRAM OR Parent Outreach Program START FOLLOW-UP page 1 of ,1 
ORGAN. Sheehv COMPONENT Hr?P'=^/P^r-Pni- r.nmpnnpni- DATE ^^p^iqyv DATE June 1978 


GOAL AT'TAINMEflT GUIDE 






SCALE ATTAINMENT 
LEVELS 


SCALE 1 : Target 
(Wp 50) Specificity 


SCALE 2 : Home-Task 
(W^^ 50) Utilization 


SCALE 3 ; DTB 

(W^= 20) Uti lization 


SCALE 4 ; New Task 
(W/^= 10 ) Information 


SCALES :Specia1 
(Wj= 20 ) Home-Time 


4. 9 

T* Ci 

Best Anticipated 

Educational 

Outcome 

Thought Likely 


More than Q0% of 
part i c i pat i ng 
parents 


More than 80^ of 
Home-Tasks 
*★ 


More than 75^ of 
participating pa rents 


More than 80^ of 
parti CI pa c 1 ng pdrenL^ 


More than 60% of 

pdl LIUipaLMiy 

parents 


More Than 
Eoq^ected Level 
of Educational 
Success 




\r 


1 






0 

Expected Level 
of Educational 
Success 


Q0% of, part i c i pat i ng 
parents will have 
identified & specifiec 
a target problem usinc 
Behav ioral Analysis 
and record problem on 
back of Parent Check- 
J ist bv June 1978. 


Par t i c i pat i ng parent 
will have taught 80^ 
of Home-Tasks as 
recorded on PEWRG 
Vi 1 i , by June 1978. 


-/S% of par t i ci pat i ng 
parents will have 
used DTB ' s as recorde( 
on PEWRG X! , by June 
1978. ' 




Q0% of parents will 
have asked for or 
offered task infor- 
mation as recorded 
by PEWRG XV, by June 
1978. 


60% of pa rt i c i pat- 
i ng pa rents VN/i 1 1 
have set aside 
special non-task 
t ime to tal k wi th 
the i r child as 
recorded on PEWRG 
XiV. bv June 1978. 


Less Than 
Expected Level 
of Educational 
Success 


50^ " 73% of parents 


60^ - 79% of Home- 
Tasks 


j 

50% - 7^% of parents ! 50% - 79^ of parents 

1 
1 
i 


40% - 59% of paren 

! 


— Ct 

Most Unfavorable 
Educational 
Outcome Thought 
Likely 


Fewer than 50^ of 
participating parents 


Fewer than GOZ of 
Home-Tasks 


1 Fewer than 50% of 
pa rt i c i pa t i ng parents 


1 

Fewer than 50% of 
pa rt i c i pat i ng parent 

i 


Fewer than kO% of 
s pa r t i c i pa t i ng 
parents 


PROPOSED PLAN 
OF ACTION 

• Q 28 
ERLC 


1 n-serv i ce PE ' s in 
Behav ioral Anal ys i s 
by consu 1 tant . 

in-service of parents 
by PE's. 

Table 7: Goal Attain 


Ongo i ng mon i tor of 
PEWRG . Susta i n parent 
pa rt i c i pat ion. 

nent Guide for the Hon 


Ongoing monitor of 
PEWRG. Sustain paren 
part i c i pa t i on . 

13/Parent Component. 


i 

tj Monitor PEWRG. 

1 

! 


Ongo i ng mon i tor 0: 
PEWRG. Sustain 
parent participa-" 
ti on. 

2 x] 
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Scale 2: Home-Task Utilization: This scale was an attempt to 

gauge the persistence with which parents used Home-Tasks. 
As recorded on the Parent Educator's Weekly Report Guide 
(PEWRG, entry VIII) every parent that committed them- 
selves to using Home-Tasks subsequently reported using 
the tasks with their child. The comment of a Parent 
Educator once again provides a relevant characteriza- 
tion: "l/ 1 could get through the door, I could get 
them to use the Home-Tasks." 
Scale 3: DTB Utilization: The Parent Educators Weekly Report 

Guide, entry XI recorded the Desirable Teaching Behaviors 
used by the parent during a given week of activity. It 
was anticipated that 75% of participating parents 
would have utilized some DTB's in their teaching. The 
record shows only 54% (n=21) of the families reporting 
the use of DTBs. Again, POP results fell short of 
expectations due to the proportion of families seen for 
counselling, and other reasons described above. It is 
recommended that future cycles of program effort con- 
centrate more energy in facilitating parental use of 
these didactic principles, not only as they apply to 
Home-Tasks, but also as they apply to ordinary, every- 
day activities occurring in the home. In this way 
families could come to capitalize on the natural environ- 
ment as a source of learning and educational stimulation. 
Scale 4: New Task Information: This measure was used to determine 
the extent to which parents began assuming the role of 
colleague. POP was moderately successful in getting 
51% (n=2(>)^f all participating parents of offer infor- 
mation or suggestions for a new task, or for extension 
of aniold task. Again, this result fell short of 
expectations due to the proportion of families seen for 

3n7^sk reasons. It is, therefore, recommended that 
future program cycles incorporate all aspects of parent 
collaboration into this measure. 
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Scale 5: Special Home-Time: This final objective was intended 

to determine generalization of parent/child interactions. 
That is, parents were asked if they had begun to set 
aside special time to talk with their child other than 
the time allocated for completing Home-Tasks. Fifty 
four percent (n=21) of participating parents indicated 
they had set aside such time during the school year. 
This would seem to be an important outcome of program 
fef forts. 

Parent Reaction Sheet 

In addition to the measures summarized on the Goal Attainment Guides, 
parents were asked to respond to a questionnaire in May of 1978. English or 
Spanish versions of the anonymous questionnaire were delivered to each family's 
home with a stamped, addressed^ return-envelope. Eleven families replied. 
These 11 responses represent approximately 50% of the Home-Task families^ remain- 
ing in the program (of the 39 POP families, 4 had moved, 3 had barely begun 
the program, 9 were being seen for non-task reasons, leaving 23 specifically 
Home-Task families.) The resu^s presented in Table 8 indicate a strong 
showing for POP in the minds ^of replying parents. Response to question 8 
indicates moderate generalization from Home-Task to school work, while questions 
9 through 12 point to enhanced functioning at home for parents and children. 
Only four parents made comnoents. They are: 

"POP helped me understand a lot about what rny child was doing in 
school. The reason I will not be continuing POP is that I will be 
working a full-time job." 

"Questions 3 and 4. I didn't have any problems following these 
teaching methods. Because I was already using them. These are 
good methods to have." 

"I feel that this program really helps children do better in 
school. And I am very glad that you have this program. Thanks 
a lot for helping my children learn better." 

"Hope the program continues, sorry she wasn't in it at an earlier 
age." 

Stanford Achievement Test Scores hf Parent Outreach Children 

One may wonder why SAT scores were not scaled using Goal Attainment Guides 
as were other outcome measures. The reason is that scaling would simply have 
made it too easy to camouflage the project's lack of experimental control. The 

3i 
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SHEEHY SCHOOL 



1^ 



PARENT POP INDEX 

(Place an-^in the column indicating your 
reaction to the question) 



1. Was it convenient to have a Parent Educator come 
to your home in the beginning? 

2. Is it convenient to have a Parent Educator come 
to your home now? 

3. How did you feel about using the 10 Desirable 
Teaching Behaviors in the beginning? 

h. How do you feel about using the 10 Desirable 
Teaching Behaviors now? 

5. How appropriate were the Home-Task for your child? 

6. How clearly were the Home-Tasks written? 

7. How well were the Home-Tasks explained^ by the 
Parent Educator? 

8. How well were the Home-Tasks able to help your 
child in doing better at school? 



9. Hew often was your child w i 1 lijiq to practice 
Home-Tasks with you in the beg i rmti 

10. How often is your child willing to practice 
Home-Tasks with you now? 

11. Was finding the time for doing Home-Tas^s with 
your ch i 1 d d i f f i cu 1 t? 

12. Has working on Home-Tasks made it easier to 
communicate with your child in other ways? 

13. Would you recommend POP to other parents? 

l**. Would you be willing to continue with POP for 
another year? 

OTHER COMMENTS: 



(NO NAME PLEASE) 
Date N = 1 1 



PRETTY 
GOOD 



55^ 



33% 



10% 



(>h% 



50% 



SS% 



ALWAYS 



36% 



60% 



ALRI GHT 



h5% 



67% 



60% 



30% 



36% 



h2% 



50% 



36% 



NOT SO 
GOOD 



0 



3% 



OFTEN 



h5% 



NOT 
OFTEN 



18% 



NEVER 





YES 


NO 






27% 


73% 






^1% 


9% 






An 


3% 






91% 


9% 





ERIC 



Please return this form, filled cfut, in the accompanying postage-paid, self- 
addressed envelope. 

32 

Table 8: Parent response to Parent Outreach Program efforts, May 1978. 
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above results obtained with Goal Attainment Guides are an adequate means to 
approach the evaluation task when the question asked is something akin to, "Is 
* the program producing and is it achieving its objectives" (House, 1978). 
These managerial questions of productivity and accountability are important 
considerations for'program staff, and goal scaling helps sharpen observation 



GAS evaluational res ul ts--many caveats were interwoven into the results 
discussed above. But, with test Scores being the social symbols of numerical 
finality they are, it would be tempting to attach causal explantions to GAS 



outcomes. It would be convenient to forget that Parent Outreach was unable to 
exercise any experimental control through randomization, and it would be 
bothersome to recall that the statistical calculations required to derive a 
GAS score did absolutely nothing to improve on statistical control. 

, Nevertheless, SAT scores do exist for treated children for both a 
September 1977 and a May 1978 administration. These scores were also used 
as the main criterion in the selection of referrals. An important question 
remaining, then, is how to report these SAT scores in a manner that is not 
misleading. The mind pleads for closure. ^ 

The scores could be treated according to one of the "pre-experi mental " 
designs cited by Campbell and Stanley (1963). But considering the weaknesses 
of these designs, and the many competing hypotheses that can be mustered to 
counter their claims of causality, it seems best to simply state pre and post 
test scores. This is done in Table 9. 

The reader who chooses to compute pre-post difference scores and interpret 



the results in some^favored manner is again referred to Campbell and Stanley (1963, 
pp. 7-12) for a dis^cussion concerning the many sources of internal and exter- 
nal invalidity of such interpretations. The reader who favors the use of 
matched controls should be made aware that classroom teachers were unable to 
find equivalent matches for 8 of 25 POP children even after the five matching 
variables of grade, sex, race/ethnicity, socioeconomic status, and Fall 
achievement level were reduced to only three. More crucial to the issue are 
the serious criticisms of matching cited on pages 2, 15, 49, and 70 of the 
Campbell and Stanley book. 



dependent decisions. Furthermore, we arrive at no spurious conclusions pro- 
viding we do not err by imputing hypothesis-discriminating ^ilities to our 
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Table 9 

Stanford Acliievement Test outcomes for 23 
Parent Outreach children at two points in time 



CHILD 


SAT Percentile Rank 
September 1977 


\ SAT Percenti 1 e 
May 1978 




8 


6 




12 


12 




14 


10 




1 


not available 


^6 


28 


22 




32 


6 




28 


58 




34 


34 




48 


28 


-^10 


12 


6 




12- 


4 


r 

"■iz 


8 


10 




4 


26 




14 


4 


^15 


8 


10 




22 


36 




20 


14 




4 


18 


^9 


8 


20 


''20 


26 


4 




30 


54 


c 

^22 


6 


12 


r 

^23 


30 


26 



Program Costs 

'Parent Outreach functioned with three Pareht Educators and one coordinator. 
The cost of a maintenance program, once devel opm^nt^^ design procedures are 
established, are given in Table 10. 

These costs c^n be related to program services in various ways. For 
instance, $14,880 represents an average expenditure of $381.54 for each of the 
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Table 10 

Costs of a Parent Outreach maintenance program 



3 hour p^iraprofessional PE 

3 hour paraprofessional PE 

2 hour paraprofessional PE 
1/4 time coordinator 



$ 4,800.00 (salary + fringe) 
4,300.00 (salary + fringe) 
2,000.00 (salary + fringe) 



5,000.00 (salary with no fringe) 



(program and caseload 
supervision , but not 
initial program develop- 
ment or evaluation 
design and summary) N 



\ 



Materials, conferences, 
mi leage 



1 ,000.00 



Subtotal 



$17,100.00 




$14,880.00 



2,220.00 



39 fami lies Served over the academic year, or $25.66 for each of the 580 family 
weeks of service. Costs also represent an expenditure of $13,46 for each of 
the 806 direct parent contacts completed over the year, or $58.68 per hour of 
direct contact with parents for each of the 253.6 hours spent in parent 
conferencing and training. These figures of course, spread costs over only 
parent contacts and do not take into consideration preparation time, or the 
innumerable hours spent directly with children either in supportive counseling 
or in classroom observations. No systematic record was kept for chi 1 d-contact- 
tims since the original intent was to work with children only through their 
parents. Reality once again overwhelmed our intentions. Future program 
cycles should record child-contact-time. 

The above figures would take on more relative meaning if they could be 
compared to programs similar to POP. The National Home Start Evaluation pro- 
vides some data that may be used for comparison (Love et al., 1976). Although 
Home Start differs from Parent Outreach in many significant ways (especially 
with regards to the age level of children), program activities and home visit 
strategies are quite similar. 

Home Start reports that, "The cost of serving a family for one year 
averaged $1,750 ($1,400 in federal funds and $350 in locally contributed goods 
and services) but ranged from a low of $1,325 to a high of $2,505" (Love et al . 
1976, p. 19). The $1 ,750 aC^erage represents a monthly expenditure of $145.83 
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per family, or $1,312.50 per family over a nine month period. In contrast, 
Parent Outreach averaged $381.54 per family over the nine TOnth academic year. 
Although the reader is again reminded that the two programs are not strictly 
comparable, these figures do seem to represent the limited amount of data 
currently available. 

A final point with regards- to costs concerns the family caseloads of 
both Home Start and Parent Outreach. Home Start determined that a full-time 
home visitor could effectively carry between 9 and 13 families; this allowed 
for approximately 3 monthly visits of 90 to 120 minutes in duration. These 
values appeared to be optimal for a home-visiting preschool program (Love et 
al., 1976, p. 28). Parent Outreach on the other hand mounted shorter but 
more frequent home visits per family (see "Results" above). In addition, 
POPs 39 families were distributed over three half-time Parent Educators 
(officially, PEs worked less than half-time on POP activities), giving each 
PE an average caseload of 13 families. The implications of these contrasts 
are not yet clear. 



CONCLUSIONS AND RECOMMENDATIONS 

« 

Based on the above results, it is concluded that the Margaret Sheehy 
School Parent Outreach Program was successful in establishing a cost-effective 
home-based school program focusing on both academic and related needs of 
refer^^d children. In addition, judging by questionnaire results and anecdotal 
comments made to PEs, it is likely that the siblings and parents of focal 
children gained in various ways from program efforts. 

Experience tells us that it is too early to detect program resultant 
changes in SAT scores^ especially considering the Large proportion of time 
required to foster a sustained home study regimen.' But, it would be prudent for 
Parent Outreach to develop an experimental or quasi-experimental design that 
would be able to legitimately utilize SAT results in future program cycles. 
It seems likely that program efforts will be reflected in enhanced achievement 
outcomes once parent/child study "takes hold" in the home for any prolonged 
period of time. 

One strategy that would make SAT scores subject to more valid interpretatiqj? 
would be the use of some variation of time-series analysis (Glass, Wi 1 1 son 
Gottman, 1975). Another strategy would be the use of "waiting list controls." 
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This strategy requires that a pool of referrals be established by the school's 
referral team, vnth families being selected for treatment on a purely random or 
chance basis. Unselected families become a randomly formed "v/aiting list control 
group." Random selection represents the 1 eas t discrimi natory (Democratic if you 
like) manner in which to assign limited and unproven resources to the 120 families 
that have already been targetted as possible POP recipients for the 1978-1979 year. 
Additionally, such a procedure is one of the few ways operable on a school site 
that can give program staff the data from which to establish "if... then" relation- 
ships. If the program staff, the school, or the community wish to make statements 
concerning program causality, then each of these spheres must assume the "risk" 
of allowing a program to function that is designed specifically to meet acceptable, 
research-oriented standards. Without such "risk," conclusions at the desired 
level of "certainty" will not be forthcoming. This does no discredit to an 
evaluation approach, however. Interestingly enough, a research-orientation may 
just possibly benefit from prior use of an evaluational-orientation, at least 
one such as the type represented by GAS. That is, several program cycles using 
GAS as a means of refining program inputs could greatly attenuate the conspicuous, 
unreliable homogeneity of the "independent variables" used in intervention research 
(Brim, 1959, p. 132; Snow, 1974), and enhance the meaning of any "if, ..then" - 
conclusions arrived at. 

Considering the high proportion of culturally non-dominant families served 
by POP in 1977-1978 and also targetted for service in 1978-1979, a word is 
necessary with respect to cultural assimi 1 ati^on. It has been mentioned that 
Home-Tasks, Desirable Teaching Behaviors, and the rest are primarily solutions * 
of the dominant culture for the perceived problems labelled "academic difficulty," 
Even though participation in POP is voluntary, it is nonetheless appropriate to 
assume a posture of sensitivity towards the interpersonal communicative patterns 
of families. Horowitz and Paden ( 1973) point out thaf once outsiders begin 
prescribing parent/child interaction patterns, ^'t no longer becomes possible 
to avoid questions of values and cultjjral relativity. For "...it can be argued 
that subcultures retain their identity in the subtle and complex patterns of 
interpersonal interactibn that one generation teaches another" (p. 354). The 
extent to which intervention programs emphasizing skills development (school 
related skills, DTBs, etc.) either hinder or enhance interpersonal processes T§ 
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not known. We would do well to heed the reminder that sensitivity to these 
issues is mandatory if we wish to contribute constructive solutions to society's 
problems (Horowitz and Paden, 1973, p. 354). To this end, a study is now in pro 
gress that seeks to describe the home learning and teaching styles of families 
representati ng an array of socioeconomic levels and racial/ethnic group member- 
ships, and whose children score across all achievement levels (Nicassio, 1977b). 

A final word is warranted concerning the use of Goal Attainment Scaling* in 
the milieu of a school. At the outset, GAS was conceived of not only as a means 
of evaluating program outcomes, but also as- a training device for the program's 
paraprofessional staff. The plan was to model and use a regenerating process 
that relied on empirical results as the basis for decision making--a process 
consonant with the prevailing, and useful, educational concepts of "diagnostic- 
prescript! ve teachi ng^" "mastery-learning," and "informational feedbac/." Because 
scaled Goal Attainment Guides were posted in open v-iew in POP's working and meeting 
area at the beginning of the school year, they provided a reference and focal 
point during team meetings. This function is illustrated by the following 
observations : 

1) Scaled goals provided a representation of initial thqughts 
/ concerning program potential against which to judge ohgoing 

1^ progress. 

\ 2) Guides provided a standard for program decisions in a 

school environment characterized by flux, shifting frames 
of reference, changing circumstances, and pressures of the 
V moment. 



\ 



3) "Proposed Plan of Action" notations provided a constant 

reminder of the techniques that were trying to be developed 

and perfected. Such reminders became important during occasional 

moments of frustration and disappointment. 

Maintaining program continuity in the face of staff mobility, a reality in 

many educational settings, can be a difficult task. When a new coordinator 

began planning th^ program for the 1978-1979 academic year, they benefited from 

the experience of the previous cycle of program efforts . And because they had 

been easily trained in the Goal Attainment Process and in the interpretation of 

outcome scores, they were able to formulate performance criteria at levels \ 

more closely and realistically matchi ngboth the strengths and constraints of ^ 

the program they inherited. Because Goal Attainment Scaling is continuing to 

be used, this coordinator will be able to pass on empirically useful results to 

& subsequent program cycles. In 'his manner, increasing quality and continuity of 
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services are ™r^ „fee,y to be assured the children and families who are the rea, 
concern of Parent Outreach. 

This paper can conclude with a quote from the National Home Start Evaluation 

h! . ' " ' '"'P'-^^lo" this writer 

Us developed after his association with the children, parents and t 

the Parent Outreach Program: ents , and stjff of 



. their'prog?^rex::"rien1:"^r^ uSr^^rded^VI'l^r^ ' 
ways that parents aooroarh rr^nfin ° ^ ""^^ Persist in the 

housing, Jo'bs, or a'm y le* "";„"\^" J:?;!'""^ P^'""--^ "^th 
new jobs and learn new sk lu I willingness to tackle 

ask for help on'therr"o ' w^e *'hey' TeZM" 'I '° 

parents, the influence nf HnmL cfi:? '^^^ ^^"^ "^a^y 

Of the relation"s^,Tb"^:e"°^e Til IVZl' i"rett^"';'^ 
more clearly understnnH h\y cnm n cmiarei;)— a relationship 

and more rewarding tSevsa"' ' 



(Love et al., 1976, p. 12) 



ERIC 



3u 



32 



REFERENCES 



Alkin, ;i. C. Evaluation theory development. Evaluation Comm e nt 2 , No. 1 , 1969. 

Brim, 0. G. Methods of educating parents and their evaluation. In 0. G. Brim 
(Ed.), Education for child rear i ng . New York: Rusiell Sage Foundation, 
1959, pp. 122-141. 

Brophy, J. E., and Good, T. L. Teacher-student relationships: Caus es and con- 
sequences . New York: Holt, Rinehart and Winston, Inc., 1974. '"^ 

Byassee, J., and Tamberino, E. Individually tailored behavioral goals and 
therapeutic summer camping: A preliminary evaluation. Goal Attainmen t 
Review , 1975-76, 1, 71-78. 

Campbell, D. T., and Stanley, J. C. Experimental and quasi-experimental 
designs for research . Chicago: Rand McNally, 1963. 

Garwick, G. Comments on Stoudenmire and Comola's application of standardized 
Goal Attainment Scales. Applications of Goal Attainment Scaling , Program 
Evaluation Resource Center, 1976, pp. 72-75. 

Garwick, G. T-Score to Percentile Rank Conversion Table: Techni cal__Regort of 
the P roject Evaluation Project . The Project Evaluation Pro^ject, ~ 
501 Park Avenue South, Minneapolis, Minnesota 55415. 



Glass, G. v., Willson, V. L.,^and Gottman, J. M. Design and analysis of time - 
series experiments , Boul'der, Colorado: Colorado Associated University Press, 
1975. 

Gordon, I. J. Early child stimulation through parent education. (Final Report 
to the Children's Bureau, Social and Rehabilitative Servi ces , Department 
of^^ealth, Education and Welfare). Gainesville, Fla.: Institute for the 
Development of Human Resources, University of Florida, 1969. 

Hegion, A. E., Fish, E., and Grace, S. Goal Attainment Scaling: An alternative 
for measuring academic progress. Goal Attainment Review , 1974, 1^, 20-34. 

Hersen, H., and Barlow, D. H. Single case experimental designs: Strategies 
for studying behavior chcinge . New York: Pergamon Press, 1976. 

Horowitz, F. D. , and Paden, L. Y. The effectiveness of environmental interven- 
tion programs, in B. M. Caldwell and H. N. Ricciuti (Eds.), Review of 
/ child development research (Vol. 3): Child development and social policy . 

Chicago: University of Chicago Press, 1973, pp. 331-402. 

House, E. R. Assumptions underlying evaluation models. Educational Researcher , 
1978, 7, 4-12. 

Kiresuk, T. J., and Sherman, R. E. Goal attainment scaling: A general method 
for evaluating comprehensive community mental health programs. Communi ty 
Me?.tal Health Journal . 1968, 4, 443-452. 



ERIC 



40 



33 



Kirkhart, K. Evaluation of a training program for adoption and foster care workers 
by Goal Attainment Scaling. Goal Attainment Reviev/ , 1977, 3, 167-192. 

Love, J. M., Nauta, M. J., Coelen, C\ 6., Hewett, K. , and Ruopp, R. R. National 
home-start evaluation: Finding? and implications. (Final Report). 
YpsiTanti, Mich.: High/Scope Educational Research Foundation, and the 
Department of Health, Education, and Welfare^ 1976. 

Nicassio, F. J. Parents as home-based tutors of their own children: A review 
of elementary level and selected preschool interventions. Unpublished 
manuscript. University of California. Santa Barbara, 1977a. 

Nicassio, F. J. Parent/ child communicative interaction patterns induced by a 
home learning task: A proposal for research.. University of California, 
Santa Barbara, 1977b. 

Nicassio-, F. J. Exercises in classroom problem solving: A short handbook in 
^ behavioral analysis and intervention. Santa Barbara: University of 
.California Extension, 1977c. (Mimeograph) 

Nicassio, F. J. Parent outreach program: Parents as home-based tutors of their 
own children. (Final Report) Merced, Cal . : Merced Ci ty School District, 
1978. 

Patterson, G. R., and Gullion, M. E. Living with children: New methods for 
parents and teachers . Campaign, 111.: Research Press Company, 1971. 

Popham, W. Jw» Educational evaluation ,. Englewood Cliffs, N.J.: Prentice-Hall, 
1975. 

Rosenthal, R. , and Jacobson, L. Pygmalion in the classroom: Teacher expectation 
and pupils' intellectual development . New York: Holt, Rinehart and 
Winston, 1968. 

Snow, R. E. Representative and quasi-representative designs for research in 
teaching. Review of Educational Research , 1974, 44, 265-291. 

State of California. Report on .the Reading Test : Second and third grades: 
Merced City Elementary: Margaret Sheehy Elementary. Sacramento, Cal.: 
California Assessment Program, 1977. 

Stelmachers, Z. , Lund, S., and Meade, C. Hennepin County Crisis Intervention 
Center: Evaluation of its effectiveness. Eval uation , 1972 , J_, 61-65. 

Stoudenmi re , J., and Comola, J. Evaluating Camp Climb-up: A two-week 

therapeutic camp. (Unpublished Report) Tupelo, Miss.: Mental Health 
Complex, 830 South Gloster Street, 1972. 

Stufflebeam, D. L., etal. Educational evaluation and decision making . Itasca, 
IlK: F. E. Peacock, 1971. 

Webb, L. Administrative evaluation in an outpatient mental health clinic 

for adolescents. Goal Attainment Review, 1975-76 , 2^, 157-168. f 



or a^" 

ERIC 4l 



\ 



Weiss, C. H. Evaluation research: Methods of assessing program e ffecti veness 
Englewood Cliffs, N.J.: Prentice-Hall, 1972. ' 

Yakima Public Schools. Project home base: Report to United States Office of 
Education. Mimeographed. Yakima, Wash.: Yakima Public Schools, 1975. 

Yakima Public Schools. Project home base: Helping parents teach their own. 
Yakima, Wash.: Yakima Public Schools, 1976. 



I 

4^ 



Footnotes 

^This paper describes the results of the labors of the following persons 
who are gratefully acknowledged: former Sheehy School principal Dr. Neil 
Schmidt who was the' inspiration behind Parent Outreach; the faculty and 
School Advisory Committee, too numerous to mention, whose budgetary 
allocations gave the program actual substance; and especially the POPPEs , 
Marge Perez, Judy Sabala, and Ola Winzer whose sensitive industry contributed 
most to the results herein discussed. The work described in this paper was 
completed while the author was a consultant and doctor'al candidate at the 
University of California, Santa Barbara. 



